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Abstract 

The development of an instrument to measure creative 
teaching abilities, ^the Creative Teaching Dilemma (CTD) , invqlved 
three phases. The instxfument was constructed and refined, and scoring 
procedures were outlined. The activities comprising the CTD included 
defining the teaching (dilemma, gathering additional facts, 
identifying and statin^g the problem, generating solutions, selecting 
criteria, evaluating alternative solutions, and stating the final 
solution. The reliabillity of scoring procedures was measured and one 
aspect of criterion-re^^Iated validity was assessed in phase two. A 
second measure of criterion validity was taken in phase three. The 
results demonstrated acceptable reliability and validity estimates. 
After comparing CTD with the Torrance Tests of Creative Thinking 
Demonstrator Form (TTCTDF), it was concluded that the two instrtjunents 
measured different aspects of the same abilities. The use of the CTD 
may provide a broader assessment of teaching performance. The 
abilities measured by CTD could be enhanced through training in 
creativl problem solving. The instrument was developed to evaluate 
facets of competency based teacher education programs. (DWH) 
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The Development of an Instrument to Measure 
Crea'ive Teaching Abilities 

John F. Riley, Ed.D. 
Lamar University 

In recent years, competency-based teacher education 
(CBTE) programs have emphasized the performance of specific, 
observable behaviors and defined competent teaching in terms 
of those behaviors. Development of these abilities may be 
an important aspect of teacher preparation. To meet the 
demands of a particular situation, a beginning teacher 
may develop a teaching style by modifying, adapting, 
elaborating, and synthesizing such, behaviors . In this 
paper, this process is defined as creative teaching, and 
the development of an instrument to measure creative 
teaching abilities, the Creative Teaching Dilemma (CTD) , 
is described. 

The Problem 

In using competency statements to evaluate the 
classroom performance of teachers, it has been pointed 

out that "a high score on any competency is no guarantee 

< 

that the teacher would have performed in this manner 

had there not been an assessment; or that the teacher will 

consis-tently perform in this manner in the future (Capie, 

Anderson/ Johnson, & Ellett, 1979; pp. 8-9)." In addition, 

classroom observation instruments are limited in the 

number of competencies that can be assessed (Capie, Anderson 

Johnson, & Ellett, 1979) . Therefore what is measured 
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by these instruments is a teacher's ability to perform 

certain competencies r durilig a given lesson. 

/ 

A typical classroom may present the teacher with a 
set of diverse, seemingly irreconcilable problems which 
require a number of diverse, perhaps contradictory solutions. 
In this setting, the teacher is called upon to apply 
previously acquired information in new ways. Torrance 

and Myers (1970) describe this process as a unique invention. 

/ 

This unique invention of the teacher emerges through 
the creative process of trying to accomplish important 
goals* As you fail or succeed in these goals, you 
become kware "of your deficiencies, defects in your 
techniq.lies and strategies, and gaps in your knowledge. 
You drajw upon your past experiences. You intensify 
your search for clues in your ongoing experiences. 
You try to apply creatively the scientifically 
developed principles you have learned in your 
professional education or through your reading. 
Then, you read and study some more. You see things 
of which you have hitherto been < unaware . You make 
or formulate hypotheses concerning ways of teaching. 
You test or modify these hypotheses and tell others 
what you have learned. (p. 102) 

-In order jbo define and measure creative teaching, 
specific creative abilities must be; identified, and assessed 
in an instructional context. For Guilford (1956) , the 
essence of creativity is divergent thinking — the kind of 



thinking in which "there is much searching or going off 
in various directions. This is most clearly seen when 
there is 'no unique conclusion" (p . 274). Divergent thinking 
consists of four factors: fluency, or production of ideas; 
flexibility, or ability to think of things in new ways; 
originality, or the production of unusual solutions to 
problems; and elaboration, the amount of detail provided 
(Guilford, 1967). Measurement of these abilities has 
provided the basis for Torrance's creativity tests, and 
for recent attempts to measure creativity in teaching. 
Two studies are of particular interest. 

In the first study, Rosenberger (1978) attempted 
to measure the effects of training in divergent thinking 
strategies on the creative thinking and classroom performance 
of student teachers . She reported significant/ treatment 
effects on the Torrance Tests of Creative Thinking (TTCT) . 
Subjects trainee^ in divergent thinking strategies had 
statistically significantly higher scores on verbal and 
figural forms of the TTCT. However, there were no 
significant differences on teaching performance ratings 
by the cooperating teacher and university supervisor. 
The study thus failed to demonstrate a connection between 
creative thinking abilities and classroom teaching performance 

In the second study, Ilutchins (1979) trained prosppctive 
teachers in Creative Problem Solving, and measured 
differences in performance on two Hypothetical problems, 
one instructional and one disciplinary in nature. Measures 



of fluency and originality, but not flexibility, were 
taken. However, no pilot study .was undertaken to indicate 
the reliability and validity of the measures. 

The shortcomings in these two studies provided the 
direction for the present study, Rosenberger ' s study 
indicates that an intermediate measure, one that is 
related to both creative abilities and classroom teaching 
performance, is needed, ilutchins recognized this need, but 
did not provide empirical data regarding the reliability 
and validity of her measures. The instrument described 
in this paper was designed to provide a/ reliable measure 
which demonstrated criterion-related validity; i,e,, 
significant relationships with measures of creative thinking 
and with measures of classroom teaching performance. 
Therefore, the following research question guided the 
development of this instrument: 

V7hat is the relationship between scores on the Creative 
Teaching Dilenuna and scores on measures of creative 
thinking and of classroom teaching performance? 

Methods 

The development of this instrument involved three 
phases. In Phase One, the instrument was constructed and 
refined, and spooring procedures outlined. In Phase Two, 
reliability of scoring procedures was measured, and one ; 
aspect of criterion-related validity was assessed. 
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In Phase rThree, a second measure of criterion- related 
validity was taken. Each phase of the study is described 
next. 
Phase One 

The activities in the CTD v^re based in part on an 

exercise entitled "Application Test: Creative Learning 

i 

and Teaching," developed and used by Torrance in his 
classes in creative thinking. The CTD consists of 
eight activities. Each of these activities is described 
in brief. 

1. The Dilemma .^ Subjects are asked to read a one-page 
description of a /liypothetical situation faced by an . 
imaginary beginning teacher. The dilemma was constructed 
|by interviewing teachers, university professors, and 

s 

undergraduates in education, regarding their experiences 
with problems^ in the classroom,. From the problems 
supplied by these resources # a narrative was composed. 

2. Additional Facts . After reading the dilemma, 
subjects are given the opportunity to supply any missing 
information that they believe' would be helpful as they 
try to solve the dilemma. This activity is designed to 
allow subjects to individualize the situation to some 
degree, based on their perceptions and experiences. It 
is intended that they supply facts, not additional 
problems or possible solutions at this point. 

3. Problem Identification * In this activity subjects 
arc given five minutes to list as many problems as they o 



can from the dilemma* They are allowed to refer to the 
dilemma in Making their lists. The time limit was 
imposed to facilitate scoring of this activity. 

4. Problem Statement , From the list of problems ^ 
they generate, subjects select and synthesize one problem 
that forms the essence of the dilemma. This problem should 
be stated as a question, so that solutions will consist of 
answers to the qusstion. 

5. Generating Solutions , For ten minutes, subjects 
list as many alternative solutions as possible to the 
problem they have stated. Again, the time limit was impo^sed 
to facilitate scoring of this activity. 

6. Selecting Criteria , Subjects select five criteria 
with which to evaluate their solutions. They are encouraged 
to select criteria that will help them discriminate 
between solutions, and that are relevant to the teaching 
situation. 

7. " Evaluation of Alternative Solutions . The five 
most promising solutions are selected informally for 
evaluation by thfe subjects. Each criterion is used to 
rank all five solutions, with a score of 5 given to the 
best solution according to that criterion, down to a 
score of 1 for the poorest solution on that criterion. 
After all criteria have been applied, totals for each 
solution indicate the overall evaluation. 

8. Statement of the Final Solution . Based on the 
evaluation, subjects wdther restate their best solution, 

8 



or combine ideas into an "even better" solution. 

For the purposes of the prcisent study, only the 
Problem Identification and Generating Solutions activities 
have been scored. Procedures for scoring other actitities 
provide a direction for future research. Two dimensions, 
fluency and flexibility, are scored for each activity. 
In each case, the fluency score is -the total number of 
appropriate, unrepeated responses within respective 
time limits. Problem fluency isi a measure of a subject's 
ability toUdentify and state pi^oblems within a situation, 
solution fluency measures the abiiity to produce alternative 
solutions to a stated problem. 

Flexibility scores are measures of shifts in thinking, 
■ and the ability to approach a situation from different 

points of view. Problem flexibility indicates the ability 

■ 1 

to Identify different kinds of problems in a situation, 
and solution flexibility measures a similar ability in 
proposing alternative solutions. 

, A preliminary form of the CT^D was ad.ainistered to ten 
graduate students in education at the University of Georgia. 
Following this administration, they discussed their 
reactions with the researcher, and made suggestions for 
the modification of the CTD. A number of changes were 
made in the directions and organization of the instrument 
as a result of this information. 

Next, scoring procedures for each dimension of the 
r^r.?^. CTD Were developed. Fluency scoring involved determining 
cKJC S \ ' 
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the number of responses to each activity* A potential 



problem involved scoring when subjects responded in 
paragraph or narrative form rather than with a list. 
In this case, each new idea was scored as a separate 
response. In addition, when subjects provided several 
responses under one major heading, as in an outline, 
each was scored as a separate response. 



a system for measuring shifts in thinking and different 



categories for responses, and derived flexibility scores 

by counting the number of categories used by each respondent. 

This procedure was followed here. 

A complete list of problems in the CTD was submitted 
to a panel of experienced teachers, each holding at least 
a Masters degree and having at least three years of 
classroom teaching experience. Each judge was asked to 
categorize the problems. With no limitations on the number 
Qf categories or the size of each category. From the five 
independent groupings, a set of eleven problem categories 
was developed, using the "best fit" of all five. The 
judges approved the final problem categorization scheme. 

In order to follow a similar procedure in developing 
solution flexibility categories, a body of alternative 
solutions was developed from preliminary administrations 
of the CTD. Solution categories were developed in the ^ 
same manner as problem categories. Thirteen solution 



Scoring flexibility involved 




approaches to a situation. Torrance (1974) has used 
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categories emerged from this process. 

. Phase Two \ 

In Phase Two, the interscorer reliability, of the CTD 
was assessed, land the relationship between CTD scores and 
teaching performance was investigated. Using the categories 
developed in Phase One, complete ^scoring instructions for 
the CTD were developed. Five raters were given written 
instructions for scoring the fluency dimension of the CTD. 
Each rater scored five tests selected at random from a 
pilot sample. Each rei^ter assigned a fluency score to the 
Problem Identification and Generating Solutions activities 
of the CTD* Scores were correlated using the Pearson 
product-moment correlation. 

The same five raters were given written instructions 
for scoring the flexibility dimension of the CTD. Each 
rater scored five tests selected at random from the pilot 
sample, and assigned a flexibility score to the Problem 
Identification and Generating Solutions actwities. The 

^ scores were correlated using the Pearson product-moment 
correlation. 

In addition to correlating flexibility scores, a 
measure of the reliability of the categorization system was 
taken, using the percentage of agreement for each response. 
Agreement in this case means that the response is placed 
in the same flexibility category by different raters. 

The second goal of this phase was to investigate the 
predictive validity of the CTD for teaching performance. 

ErIc n 
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specifically for classroom management and problem solving. 
The measure of teaching performance used with both student 
teachers. and beginning teachers in the state of Georgia 
is the Teacher Performance Assessment Instrument (TPAX) , 
a classroom observation instrument. Its reliability and 
validity have been well-documented (Capie, Ellett, & 
Johnson, 1979; Ellett, Capie, & Johnson, 1980). V^ithin 
the TPAI,-a classroom managoment factor has been Identified 
(Tobin, Capio, Ellett, & Johnson, 1900) , and thi^ factor 
was used as the indicator of ability in classroom nanaqement 
and problem solving for this study. From the set of factor 
score coefficients derived from orthogonal factor analysis, 
a classroom management score may be derived fo3: a subject. 

Classroom management scores were derived in this manner 
for 33 student teachers in Early ^Childhood Edxication (K-4) 
at the University of Georgia • Ratings of these student 
teachers were made by a university supervisor trained in 
the use of the TPAI. Using the Pearson product-moment 
correlation, classroom management scores were correlated with 
subjects* scores on the CTD. 
Phase Three 

In Phase Three, the relationship between scores on the 
CTD and measures of creative thinking vas investigated* Fifty- 
eight undergraduates majoring ih Early Chilcln'6od Education 
at the University of Georgia served as subjects for this phase. 
These subjects were all enrolled in the second quarter junior 
year methods course in Early Childhood Education. 
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The Torrance Tests of Creative Thinking Demonstrator 
Form (TTCTDF) was used to measure creative thinking. The 
TTCTDF provides verbal and figural creativity scores in 
a streamlined format, using activities similar to previous 
Torrance creativity tests. One major advantage is the 
reduced time of administration — approximately 20 
minutes. Another is the possibility of gathering both 
verbal ando figural scores in so short a time. 

Verbal tasks on the TTCTDF included product improvement, 
asking questions, just suppose, and unusual uses* Figural 
tasks involve incomplete figures and repeated figures^ 

Preliminary work with the TTCTDF has produced interrater 
reliabilities consistently above .90. A number of validity 
studies are now under way. Construct validity studies thus 
far completed are promising. (Torrance, 1980) 

Subjects were administered the TTCTDF and the CTD. 
Scores were correlated using th^ Pearson product-moment 
correlation. 

Results 

Interscorer Reliability 

Five raters were. given written instructions for scoring 
each of the dimensions of the CTD. Scores were correlated 
using the Pearson product-moment statistic. Correlations for 
Problem Fluency and Solution Fluency were all l.*00. 
Correlations for Problem Flexibility ranqed from .75 to .99, 
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with a mean correlation of ^^.90/" Correlations for Solution 
Flexibility ranged, from- .64 to .87, with a mean correlation' 
of .78. Correlation matrices are in Tables 1-2. 

Table 1 

Interrater Reliability Correlations for 
Problem Flexibility Scores of the CTD 



Rater 


# 12 3 


4 


5 


1 


.89 .89 


.90 


.98 


2 


.75 


.81 


.85 ' 


3 




.99 


.95 


4 






.96 


Note: 


X„ = .90 . 








Table 2 








Interrater Reliability Correlations 


for 






Solution Flexibility Scores 








of the CTD 







Rater # 1 2 3 4 ' 5 



. 1 




.85 


•"79 


.81 


-.81 


2 




» 


.80 


.87 


.64 


3 








.78 


.73 


4 










.71 



Note: Xr = .78 • 
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Percentage of agreement in categorizing responses was 
also measured. Percentage of agreement for Problem Flexibility 
bility ranged from 65.0 to 81.7, with a mean percentage of 
agreement of 73.7. Percentage of agreement for Solution 
Flexibility ranged from 71.6 to 91.9, with a mean percentage 
of agreement of 78.6. Tables 3-4 contain the percentage^f^ 
agreement matrices. 

Relationship Betweep the CTD and Teaching Performance 

Using the Pearson product-moment correlation , the CTD 
scores of 33 student teachers at the University^ of Georgia 
weria correlated with classroom management scores derived 

i 

from the TPAI , a classroom observation instrument used to/ 
assess beginning teachers, in the state of Georgia. Four 
correlations were derived in this manner. Of the four, 
only Solution Fluency (r = .39; df =,31; n< .05) reached 
statistical significance. Correlations for all dimensions 
of the CTD are presented in Table 5. 

Relationship Between the CTD and Creative Thinking 

The CTD scores 'of 58 undeifgraduates majoring in Early 
Childhood Education at the University of Georgia were 
correlated with their scores on the Torrance Tests of 
Creative Thinking Demonstrator Form (TTCTDFi , a measure of 
creative thinking. Sixteen correlations were derived. The 
correlation between CTD Problem Fluency and TTCTDF Verbal 
Fluency reached statistical significance (r = .41; df = 56; 

ERIC 15 / 
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Table 3 

Percentages of Agreement for Categorizing 
Problem Responses to the CTD 



Rater # 




1 


2 


3 


4 


5 


1 






80.0 


71.7 


71.7 


81. 7 


2 








71.7 


65.0 


75.0 


3 










78.3 


71.7 


4 












70.0 



Note: X„ = 73,7 



Table 4 



Percentages of Agreement for Categorizing 
Solution Responses to the CTD . 



Rater # 


1 


2 


3 


4 


5 


1 




91.9 


71.6- 


81.1 


77.0 


2 






75.7 


^ 82.4 


77.0 


3 








77.0 


74.3 


4 










78.4 



Note: = 78.6 
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£<.01). The relationship between CTD Solution Fluency and 

TTCTDF Verbal Fluency was also statistically significant 

(r = .60; df = 56; £. < .-01) . 

There was a statistically significant correlation • 
between CTD Problem Flexibility and TTCTDF Verbal Fluency 

(r = .31; df « 56; £< .05). The relationship between 
CTD .Solution Flexibility and TTCTDF Verbal Fluency was 
statistically significant (r = .42; df = 56; £ .01). 
Other correlations did not reach statistical significance. 
These correlations are presented in Table 6. 

/ Table 5 



Correlations Between CTD Variables 
and Classroom Management 
on the TPAI 



I 

17 



; 

i 
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Table 6 



Correlations Between CUD Variables 
and TTCTDF Dimensions 





Problem 
Fluency 


Solution 
Fluency 


P fob lorn 
Flexibility 


Solution 
Flexibility 


TTCTDF 
Verbal 
Fluency 


.41** 


.60** 


.31* 

• 


.42** 


TTCTDF 

Figural 

Fluency 


.10 


.26 


.17 


.19 


TTCTDF 
Verbal 
Flexibility 


.12 


.17 


.21 


.17 


TTCTDF 

Figural 

Flexibility 


.23 


.19 


.^25 


-.04 



* £< ".05; df = 56 



** £ < .01; df = 56 

/ 

/ . 
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Discussion 

While the CTD has demonstrated acceptable reliability 
and validity estimates , more work with it is needed. For 
example, based on similar ^work done by Torrance (1974), 
scorers could be providecl with oral as well as wr*itten 
directions for scoring. This might improve the interrater 
reliability for the flexibility measures in the CTD. 

Dev(Slopment of scoring procedures for originality and 
elaboration could give a more* complete picture of creative 
teaching abi^lities. Originality and elaboration measures 
could be provided /for the Problem Identification and 
Generating Solution activities. Once sufficient data 
are collected, original or statistically infrecment 
responses could be determined and used to score an oriqinality 
dimension. Torrance (1968) has outlined a procedure for 
determining elaboration scores on verbal tests. It involves 
counting the number of additional details used to elaborate 
an idea, without rewarding wordiness. 

Scoring procedures for other activities in the CTD could 
also be developed. For example, points could be awarded for 
combining ideas, both at the problem statement stage and 
in the final solution. This type of scoring could be 
handled in the same way as the "Checklist of Creative 
Strengths" in the streamlined scoring procedures of the 
TTCT (Torrance & Ball, 1980). Evaluating the appropriateness 
of criteria used would present a challenge, but a measure of 
this type would be valuable in training and assessing teachers. 
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Beyond these considerations, itjis valuable to return 
to the research question used to guide tfie development of 
the CTD: 

What is the relationship between scores on the Creative 
Teaching Dilemma and scores on measures of creative 
. thinking and of classroom teaching performance? 
The relationship between oCTD scores and measures of 
creative thinking was statistically significant in four 
cases. TTCTDF Verbal Fluency was moderately correlated 
with each of the four CTD dimensions ♦ After closely 
examining both instruments, it is concluded that the TTCTDF 
called for more divergent thinking, while the CTD elicited 
thinking that was more convergent, in nature, due to the 
nature of the task. But it can also be concluded that the 
two instruments are measuring slightly different aspects 
of the same abilities. 

The Solution Fluency dimension of the CTD also 
showed a moderate correlation with a measure of classroom 
performance, the classroom management factor of the TPAI 
(r « .39). There is apparently a relationship between the 
ability to generate alternate solutions to a problem and . 
the ability to manage a classroom successfully. Again, the 
magnitude of the relationship indicates that there are 
other factors involved. Yet these two relationships, 
when taken together, imply that there is a connection 
between creative thinking abilities and classroom teaching 
performance, and that the CTD provides a first step toward 
^ measuring these abilities. On 

' ERIC / 



Use of the CTD may help to pirovide a broader assessment 
of tjeaching performance. Additional research by Riley 
(1980) indicates that the abilities measured by the CTD 

0 

can be enhanced through training in Creative Problem 
Solving and similar programs. Inclusion o.f programs of 
this nature may begin to extend teacher training and 
evaluation beyond the level of minimum competence. 
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